A clustered file system is a file system which is shared by being simultaneously mounted on multiple servers or “nodes.” Clustered file systems can provide features such as location-independent addressing and redundancy which improve reliability or reduce the complexity of other parts of the clustered file system. A distributed file system is similar to a clustered file system, but a distributed file system does not share block level access with other nodes of the file system. Both clustered and distributed file systems (collectively, and individually “clustered/distributed file systems”, or “C/DFS”) gain a performance benefit by caching reads and writes to physical storage units attached to nodes of the C/DFS. In any caching system, cache coherency must be maintained. Cache coherency ensures that a file system read operation always returns the most recent version of a file system object. Cache coherency also ensures that a write operation writes the most recent version of a file system object to storage. When a C/DFS node reads a cache, peer nodes in the C/DFS must be notified to ensure that they are accessing the most recent version of a file system object or object extent. When a C/DFS node writes to cache, peer nodes must be notified to invalidate older cached versions of file system objects.
Currently, caching is performed using volatile random access memory (VRAM). Read/write access to VRAM is fast, but VRAM is expensive in terms of cost. In a C/DFS system, the cost of cache is multiplied by the number of nodes in the C/DFS. Thus, a C/DFS must balance the cost of VRAM cache against the performance gain offered by caching storage reads and writes. In addition, volatile RAM cache is vulnerable to data loss due to power outages and unexpected, uncontrolled C/DFS node failures.
New, fast, non-volatile random access memory (“persistent cache”) storage technologies, such as 3D Crosspoint (3DXP) are coming on the market that have similar read/write access times to VRAM and are less expensive than VRAM and have access times that are similar to VRAM. 3D Xpoint (3DXP) is a non-volatile memory technology by Intel® and Micron Technology®, that is a bit storage based on a change of bulk resistance, in conjunction with a stackable cross-gridded data access array. A C/DFS would benefit from the fast access time, non-volatility, and lower cost of these new storage technologies. A challenge with providing a persistent cache tier below a VRAM file buffer cache in a C/DFS is that the persistent cache does not participate in the C/DFS cache coherency protocol for the VRAM file buffer cache. When a peer node of a C/DFS receives cache invalidations for shared file system objects, the C/DFS will not know that the persistent cache may contain an invalid version of the file system object. Conversely, when a peer node of a C/DFS receives a request to flush writes from its VRAM file buffer cache to the C/DFS, the writes may be held in the persistent cache for some time afterwards, causing the peer nodes to believe that the most recent version of a file system object (or object extent) is persisted at a physical storage device in (e.g. at a storage array) when in fact the write is only contained with the persistent storage buffer visible to the node of the C/DFS that initiated the cache flush operation.